{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "685c037d",
   "metadata": {},
   "source": [
    "# Tensors\n",
    "\n",
    "All data in machine learning is expressed as tensors. Tensors are multi-dimensional arrays."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b293baa",
   "metadata": {},
   "source": [
    "**Terminology**:\n",
    "- **1D tensor**: vector\n",
    "- **2D tensor**: matrix\n",
    "- **3D+ tensor**: tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7777cecb",
   "metadata": {},
   "source": [
    "**Index**:\n",
    "- [1. Tensor creation](#1-tensor-creation)\n",
    "- [2. Tensor data types](#2-tensor-data-types)\n",
    "- [3. Math with tensors](#3-math-with-tensors)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7a107f6",
   "metadata": {},
   "source": [
    "## 1. Tensor creation\n",
    "\n",
    "In PyTorch, there are two main ways to create a tensor:\n",
    "1. `torch.empty()` creates a tensor of the given size filled with uninitialized data. I.e., it allocates memory for this tensor shape, and does not initialize it with specific values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "df08dfae",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'torch.Tensor'>\n",
      "Tensor shape (rows, columns): torch.Size([3, 4])\n",
      "tensor([[0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0.]])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import math\n",
    "\n",
    "x = torch.empty(3, 4)\n",
    "print(type(x))\n",
    "print(\"Tensor shape (rows, columns):\", x.shape)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cec96e7",
   "metadata": {},
   "source": [
    "2. `torch.tensor()` creates a tensor with the specific values provided"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "b747a35f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor shape (rows, columns): torch.Size([3, 2])\n",
      "tensor([[1, 2],\n",
      "        [3, 4],\n",
      "        [5, 6]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([[1,2], [3,4], [5, 6]])\n",
    "print(\"Tensor shape (rows, columns):\", x.shape)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93750c16",
   "metadata": {},
   "source": [
    "*Note: torch.tensor creates a copy of the data we specify. This is because the underlying memory representation of a python list is not the underlying memory representation of a tensor.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c9eabb8",
   "metadata": {},
   "source": [
    "## 2. Tensor data types\n",
    "\n",
    "The simplest way to set the underlying data type of a tensor is with an optional argument at creation time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "69dac75f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1, 2], dtype=torch.int16)\n",
      "tensor([3.0938, 4.1875], dtype=torch.bfloat16)\n",
      "tensor([3, 4], dtype=torch.int32)\n"
     ]
    }
   ],
   "source": [
    "from torch import bfloat16\n",
    "\n",
    "\n",
    "a = torch.tensor([1,2], dtype=torch.int16)\n",
    "print(x)\n",
    "\n",
    "b = torch.tensor([3.0938,4.1875], dtype=bfloat16)\n",
    "print(b)\n",
    "\n",
    "c = b.to(torch.int32)\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea70f1e6",
   "metadata": {},
   "source": [
    "## 3. Math with tensors\n",
    "\n",
    "So, what can we do with tensors? First, let's take a look at simple arithmetic with scalars"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baf0a3ce",
   "metadata": {},
   "source": [
    "#### Arithmetic with tensors <> scalars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "2670fe7c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1.],\n",
      "        [1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "add = torch.zeros(2,2) + 1     # adds scalar (1) to every element in the tensor\n",
    "print(add)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7d72b48",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[3., 3.],\n",
      "        [3., 3.]])\n"
     ]
    }
   ],
   "source": [
    "multiply = torch.ones(2,2) * 3  # multiplies scalar (3) to every element in the tensor\n",
    "print(multiply)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "a059f02f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[16, 16],\n",
      "        [16, 16]])\n"
     ]
    }
   ],
   "source": [
    "exponential = torch.tensor(([2,2],[2,2])) ** 4  # raises every element in the tensor by a scale of 4\n",
    "print(exponential)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b80e7393",
   "metadata": {},
   "source": [
    "As you can see, arithmetic operations between tensors and scalars (ex. addition, subtraction, multiplication, division and exponentiation) are distributed over every element in the tensor."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32d52605",
   "metadata": {},
   "source": [
    "#### Arithmetic with tensors <> tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ea64d82",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([4, 6])\n",
      "tensor([[4, 6],\n",
      "        [6, 8]])\n"
     ]
    }
   ],
   "source": [
    "add_tensors = torch.tensor([1,2]) + torch.tensor([3,4])     # adds two 1D tensors elementwise: [1+3, 2+4] = [4, 6]\n",
    "print(add_tensors)\n",
    "\n",
    "add_tensors_v2 = torch.tensor([1,2]) + torch.tensor(([3,4], [5,6]))     # adds a 1D tensor [1,2] to each row of a 2D tensor [[3,4], [5,6]] via broadcasting\n",
    "print(add_tensors_v2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "632f4d53",
   "metadata": {},
   "source": [
    "Adding two tensors means adding their numbers one by one in matching positions, and if their shapes allow, PyTorch will automatically stretch them to fit."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
